Structured illumination is a lighting technique used in machine vision, one application being to optically recover object shape information. For example, in Horn, Robot Vision, MIT Press, 1986, p. 95, a system is described that uses a "sheet" of light and a CCD camera to triangulate a plurality of surface locations of an object to produce a range image. A range image is an image wherein each pixel represents a perpendicular distance from a corresponding point on the surface of an object to a reference plane, e.g., a depth or a height.
"Laser light striping" is a method of structured illumination that is commonly used to obtain range images using such triangulation methods. In laser light striping, periodic, i.e., repeating stripe patterns of illumination are used. For high speed image acquisition, area array cameras are used.
Liquid crystal display (LCD) light projectors are also used in periodic light stripe triangulation, since the pattern position or period can be altered electronically to produce higher resolution images. See, for example Sato and Inokuchi, "Range-imaging System Utilizing Nematic Liquid Crystal Mask", Proceedings of the First International Conference on Computer Vision, p 657-661, IEEE Comp. Soc. Press, 1987, which discloses the use of complementary projected patterns for binarizing an input image.
In M. Rioux and F. Blais, "Compact three-dimensional camera for robotic application". J. of Optical Soc. of America A, 3(9):1518-1521, September 1986, a different method of object ranging is disclosed that uses a structured illuminator which projects an array of dots. The illumination pattern is projected onto an object, which is viewed by an area array CCD camera. Range information is recovered at each dot location using the extent of defocus of the dot, as measured by the diameter of the defocused image of the dot. Rioux's method is a precursor of a family of "depth from focus" and "depth from defocus" methods which do not use triangulation. The methods in this family all employ the focal characteristics of a lens to determine surface range by measuring local image sharpness or local image blur. These methods may also use structured illumination having a fine periodic pattern to force texture onto otherwise uniformly reflective objects. In the case of depth from defocus, as disclosed in M. Watanabe, S. K. Nayar, and M. Noguchi, "Real-time Computation of depth from defocus". Proc. of the SPIE, vol 2599:A-03, p. 14-25, November 1995, the known spatial frequency content of periodic structured illumination has been exploited to simplify computation of degree of defocus, i.e., blur, in an image.
In machine vision, it can be advantageous to obtain both a normal reflectance image of an object, and a corresponding range image of the object. For example, when inspecting solder balls on ball grid array (BGA) semiconductor packages, it can be preferable to locate ball center positions by machine vision analysis of a reflectance image of the solder balls. Ball center heights can then be gauged at each found ball center position using the corresponding range image of the solder balls. Due to the three-dimensional nature of the objects being inspected, it is important that the angle of illumination and angle of viewing be the same when producing the reflectance and range images. If the angle of illumination is not the same, shadowing and shading will create differences in image brightness, herein termed "photometric non-correspondence". In the BGA inspection example above, shadows can shift the ball center position found in analysis of the reflectance image, resulting in position measurement error. This will cause ball height to be gauged in the wrong position of the range image of the example. Moreover, if the angle of viewing is different, surface relief will create displacements of object features between the two images, herein termed "geometric non-correspondence". In the BGA inspection example above, displacement of object features between the two images can result in correspondence errors, resulting again in errors in determining height from the height image.
Some machine vision applications require both exact geometric correspondence and exact photometric correspondence between a uniform illumination reflectance image and a structured illumination reflectance image, such as when a reflectance image and a range image must be compared, because the range image is derived from the structured illumination reflectance image. This comparison requires that the structured and reflectance images are precisely registered, i.e., geometric correspondence is required, and requires that the illumination conditions are substantially identical except for the structuring component, i.e., substantial photometric correspondence is required.
One method for obtaining images under a mixture of structured and uniform illumination uses two light sources: a structured illuminator and an unstructured illuminator. However, photometric correspondence is very difficult to obtain using this illumination approach. If the two illuminators are mounted side-by-side, i.e., not in coaxial relationship, the consequent small difference in angular displacement of the illuminators results in significant differences in object shading. A coaxial arrangement of the illuminators can be achieved using a half-silvered mirror (beam splitter) that combine the two light sources so as to substantially remove gross angular discrepancies. However, using a coaxial arrangement will not provide photometric correspondence if the illuminator efficiency of the structured illuminator and the illuminator efficiency of the uniform illuminator are different, resulting in photometric non-correspondence. Further, it is very difficult to make a highly uniform illuminator, i.e., an illuminator that provides illumination uniformly over the entire cross-sectional area of its illumination beam, and non-uniformity of one illuminator will generally not match the non-uniformity of a second illuminator. This phenomena also contributes to photometric non-correspondence.
Alternatively, it is possible to take an image acquired under structured illumination and electronically filter out the structured illumination component to obtain a uniform illumination reflectance image. However, the resulting image will have a resolution which is limited by the frequency characteristics of the filter. Consequently, the structured and unstructured illumination images will exhibit photometric non-correspondence on object features with image components of high spatial frequency, such as step edges.
Other methods can be described for obtaining both uniform illumination reflectance images and structured illumination reflectance images of an object, but difficulties remain in obtaining such images in a practical and economical manner that also exhibit precise geometric and photometric correspondence.